High-dimensional Gaussian model selection on a Gaussian design

We consider the problem of estimating the conditional mean of a real Gaussian variable Y = ∑p i=1 θiXi+ ǫ where the vector of the covariates (Xi)1≤i≤p follows a joint Gaussian distribution. This issue often occurs when one aims at estimating the graph or the distribution of a Gaussian graphical model. We introduce a general model selection procedure which is based on the minimization of a penalized least-squares type criterion. It handles a variety of problems such as ordered and complete variable selection, allows to incorporate some prior knowledge on the model and applies when the number of covariates p is larger than the number of observations n. Moreover, it is shown to achieve a non-asymptotic oracle inequality independently of the correlation structure of the covariates. We also exhibit various minimax rates of estimation in the considered framework and hence derive adaptiveness properties of our procedure. Key-words: Model selection, Linear regression, oracle inequalities, Gaussian graphical models, minimax rate of estimation ∗ Laboratoire de Mathématiques UMR 8628, Université Paris-Sud, 91405 Osay † INRIA Saclay, Projet SELECT, Université Paris-Sud, 91405 Osay in ria -0 03 11 41 2, v er si on 2 28 A pr 2 00 9 Sélection de modèles en grande dimension pour des design gaussiens Résumé : We consider the problem of estimating the conditional mean of a real Gaussian variable Y = ∑p i=1 θiXi+ ǫ where the vector of the covariates (Xi)1≤i≤p follows a joint Gaussian distribution. This issue often occurs when one aims at estimating the graph or the distribution of a Gaussian graphical model. We introduce a general model selection procedure which is based on the minimization of a penalized least squares type criterion. It handles a variety of problems such as ordered and complete variable selection, allows to incorporate some prior knowledge on the model and applies when the number of covariates p is larger than the number of observations n. Moreover, it is shown to achieve a non-asymptotic oracle inequality independently of the correlation structure of the covariates. We also exhibit various minimax rates of estimation in the considered framework and hence derive adaptivity properties of our procedure. Mots-clés : Sélection de modèles, régression linéaire, inégalités oracles, modèles graphiques gaussiens, vitesse minimax d’estimation in ria -0 03 11 41 2, v er si on 2 28 A pr 2 00 9 Model selection on a Gaussian design 3